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ABSTRACT: 



The stochastic properties of the output of a multiprogramming 
computer system are studied by means of a simple cyclic queueing 
model. It is shown that output is asymptotically normally 
distributed. The parameters are determined by considering a 
cumulative stochastic process that depends upon busy period 
properties; the latter may be recursively determined. Numer- 
ical examples are provided. 
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1 . Introduction 

It is often mathematically convenient and useful to represent 
the behavior of a computer system, or part thereof, as a single server 
queueing process. This is appropriate even when several servers are 
present, as in multiprogramming situations involving cyclic queues; 
see Gaver [ 2 ] , and Lewis and Shedler [ 3 ] . Then the server singled 
out for particular attention usually possesses ’’general" (non-exponen- 
tial) service times, while the others enjoy simple Markov- convenient 
properties. Assuming this structure it is often possible to compute 
such system characteristics as waiting time properties and server 
idleness probability, where the latter depend upon server processing 
rates and the number of programs (customers) allowed to be present in 
the system simultaneously. 

This paper is devoted to studying the distribution of the output 
of the server in such a process. By this we mean the following. Begin- 
ning at some moment t 1 the total number of service completions, during 
(t f ,t ! + t) is observed. Denote this number of Z(t T ,t ! + t) , or by 
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Z(t) if the process has stationary increments. We term Z the output 
of our server, and seek to characterize its behavior. It will be shown 
in the sequel that in interesting cases Z is approximately normally 
distributed as t becomes large. Furthermore, in cyclic models, e.g., 
for multiprogramming, a simple continuity argument shows that the out- 
puts of both servers enjoy the same limiting normal distribution. The 
methods employed make possible a comparison of various multiprogramming 
situations. Some limited numerical illustrations are presented. 
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2 . Inputs and Outputs 

In this section we record a simple observation upon which much 
of the later development rests. 

(a) The M/G/l Service System. Here A denotes the Poisson arrival 
rate, and S is a generic service time. Assume E[S 2 ] < °°. Let 
N(t) denote the number of customers in the system at time t, and 
let the input , A(t) , be the total number of arrivals to have occurred 
in (0,t). Then Z(t), the total output in (0,t), is defined by 
the continuity relation 

N (0) + A (t ) = Z(t) + N (t ) (2.1) 

Now suppose p = AE [ S ] < 1. Then, for large t, N(t) is finite while 
A(t) becomes large, and hence Z is asymptotically similar to A(t) . 
Putting this formally, write (2.1) as 



A(t) - At 




Z(t) - At 


N(t) - N (0) 


A£ 




AT 


At 



Now when p < 1 and E[S 2 ] < <» it is well-known that 

lira E[N(t) |N(0) ] = E[N (°°) ] < “ (2.3) 

and by Tchebychev's inequality. 



p(| N(t) - N jOl, > E) t ^ 0 (2.4) 

At zAt 

for any £ > 0 as t -*■ °°. Thus it follows that the left-hand side 
of (2.2) approaches zero in probability, and hence since the distribution 
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of converges to the N(0,1) law, so does that of ^ — - 

/At" /\t 

This result will hold true for many types of queueing systems, e.g., 

for the GI/G/1 as well as for various multiple server configurations. 

(b) The Cyclic System. Of particular interest in the multiprogramming 

computer system studies context is the cyclic arrangement depicted in 

Fig. 1. 



CPU | — 



-> o o PXU 



Figure 1 



In the most rudimentary model a fixed finite number, J, of tasks or 
programs is present in the system at any time. A program is processed 
at the Central Processing Unit (CPU) until an interruption occurs 
("page interrupt" in certain types of machines) for lack of information. 
At this moment the program enters the Data Transfer Unit (DTU) stage, 
where it awaits and eventually receives the required information and 
is then returned to the CPU stage. In the mean time the CPU may be 
busy processing another program, and is therefore kept busy. Programs 
that are completed at the CPU stage are assumed to leave the system 
and be instantaneously replaced. Models of this type have been consid- 
ered by various authors, cf. , Gaver [ 2 ], Lewis and Shedler [ 3 ], and 
Shedler [ 4 ] . 

Although only a limited amount of actual data analysis has been 
carried out, it is apparently roughly appropriate to assume that the 
service times of programs at the CPU are independently and exponentially 
distributed. Service times at the DTU are of non-exponential (more 
nearly constant) character. 
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We shall, as a consequence, make such assumptions; it is noted that 
this represents a reversal of the assumptions made in Gaver [2]. 

In order to discuss the outputs of the servers, denote by C(t) 
the number of programs that complete the CPU stage (number of CPU 
service completions) in (0,t), while D(t) refers to the corre- 
sponding quantity for the DTU. In the present model the programs 
remain the same and there are no actual departures from the system. 
Later, we put in a departure mechanism. Next, N (t) and N (t) 

U L) 

represent the number of programs at the CPU and DTU stages re- 
spectively. Then again the continuity relation (where C takes the 
place of Z) states that 

N c (0) + D ( t) = C (t) + N c (t) (2.5) 

Hence if there exist norming constants y and a such that 
[D(t) - yt](a/t") ^ has a limiting normal distribution, then, by the 
same argument as that outlined in connection with the M/G/l-system, 
[C(t) - yt](a/t) ^ approaches the same normal distribution. That 
is, the output distributions of CPU and DTU are asymptotically 
identical. But for the present model it is evident that D(t) is 
actually a cumulative process in the sense of Smith, cf . Cox [1], or 



Smith [5]. Hence asymptotic normality follows, e.g., from Smith's 
development ([5], pp. 262-263]). 
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3 . The Cyclic System, Its Busy Periods, and a Cumulative Process . 

The process associated with the cyclic system of Figure 1 may 
conveniently be viewed as a succession of busy and idle periods for 
the DTU. Let there be J programs circulating, and consider a 
moment t 1 such that N^Ct'-O) = 0 but Np(t ! ) = 1. That is, the 
system is idle prior to t ! , becoming busy with the service of one 
customer at t 1 . A busy period for the DTU is defined by 

t 1 (J) = inf { t ;> 0|N D (t+t') = 0} (3.1) 

The subscript indicates that one program is present at the start of 
the busy period. Following each busy period is an idle period, during 
which all J programs are queued behind the CPU. By the Markov prop- 
erty of the CPU service times, a generic idle period of duration I 
is exponentially distributed with mean A ^ . Successive idle periods 
and subsequent busy periods are independently and identically distri- 
buted random variables. Put 

X<n> . i<n> + T <">(j) for „ = 1 ,2 , . . . ; (3.2) 

/ \ 

{X^ '} represents the times between the successive regeneration 
points at which the DTU becomes idle. In terms of the {X^ , 
n = 1,2,...} sequence, which is one of independently and identically 
distributed random variables, one can speak of a renewal counting 
process, R(t) , where 

R(t) = 0 iff X^ > t 

(3.3) 

R(t) = j iff | £ t and \ X^ > t 

n=l n=l 



for j = 2,3, . . . . 
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Let the output, or number of service completions, during a (the 
nr^-) busy period be 

|^ n) (J) = D(t^ n) (J)) for n = 1,2,3 (3.4) 

Then the output process D(t) is cumulative in the sense of Smith [5], 
with 

R(t) 

D (t) a l g} n '(J) (3.5) 

n=l 



as t oo (the approximation consists in neglecting outputs during 
part of an X-cycle; these are negligible for large t) . A 
central limit theorem for such processes, cf. Cox [1], enables one 
to show that D(t), appropriately normalized, is approximately normally 
distributed for large t, and to find the parameters of the limiting 
distribution (asymptotic mean and variance) explicitly in terms of the 
CPU service rate, A, and the distribution of service times at the 
DTU. To be specific, it may be shown that as t 00 

E[e (j) ] 

E[D(t) ] ~ t — ~ — sty (3.6) 



and 

r Var [ B (J) ] Var [X] (E[ B (J) ] ) 2 
Var [D (t) ] ~ + (E[X]) 3 

Cov [ g (J),X]E[6. (J) ] 'l 
“ 2 (E[X] ) z ) (3 ’ 7) 



= O 



2 



t 



and that — - — tends to the N(0,1) 

o/t 

large. Explicit evaluation of the parameters y 



law as t becomes 
and a 2 



is discussed 



8 



in the next section. Finally the development of Section 2 then implies 

that also has the limiting N(0,1) distribution. 

a/t 
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4 . Cumulative Process Parameters 

Recursive evaluation of busy period properties as J increases 
was discussed in Gaver [2]. Let S be the DTU service time, with 
distribution U(x)and Laplace-Stielt j es transform 



u(s) 



e SX U {dx} , 



0 - 



Then consider the following cases. 
Representation . 

(A) J = 1. Here clearly 



(4.1) 



t 1 (1) = S 6 1 (1) = 1. (4.2) 

(B) J = 2. Condition on S to find that 



Probability 

-AS . -AS 

e 1 - e 




Use of the symbol 1 means that, for example, x|(2) has the t^(2) 

distribution but is independent of events leading up to the initial 

service completion. To explain further, consider the situation just 

following the initial service completion of the busy period. Either 

—AS 

(i) no CPU output occurred during S, an event of probability e 
in which case the busy period is of duration S with one output, or 

(ii) exactly one CPU output occurred, an event of probability 1 - e 
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in iich case thu initial situation r,7 ^s reproduced, with one program 
at -he CPU and one at the OTU, out wirh an initial c< nponent of busy 
period duration, S, and one init?el output. 

(C) Arbitrary J. Again condition on S. Define and 3^(J) 

(i = 1,2,...,J) to be respectively the first passage time from i to 
i - 1 and the output therein, given that a service is just commencing 
a. the moment N = i. 

For the present setup we » hen have 



Prcbc _ l ity 




Now a little reflect*^, allows that x^(J) has the same distributi n 
as T^(j-i+l), and similarly that 6^(J) has the same uj i t~ibut loi a. 
B^(J-i+l). This fact enables us to successively compute the various 
expectations required to evaluate (3.6) and (3.7). he now illustrate. 
Expectations . 

(A ! ) J = 1. Directly, 

E[t 1 (1)] = E[S], Var[T 1 (l 1 = Var[S]. 

E^Q)] = 1 , Var[6 1 (l)] = 0 (4.3) 



Cov[t^(1) >6^(1) = 0 
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( B * ) J = 2. Conditional on S, 



E[t 1 (2) | S] = S + (1-e AS )E[t 1 (2) ] 



(4.4) 



Consequently after removal of the condition on S and use of (4.1). 



E[t (2)] = ■ E[S] = 

1 E[e" XS ] u[X] 



(4.5) 



Likewise , 



E[6 X (2)] = 



r "AS. 

E[e ] 



u [ A ] 



Squaring column by column in Fig. 2 delivers second moments. 



(4.6) 

For example. 



E[t 2 (2)|S] = S 2 e XS + E[(S + tJ(2)) 2 ](1 - e XS ) 



= S 2 + { 2 S E[t ;l (2) ] + E[t 2 (2)]}(1 - e XS ). 



(4.7) 



So, upon removal of the condition on S, 



-AS. 



E[t 2 (2) ] = 



E[S 2 ] + 2E[S(1 - e )]E[t 1 (2)] 



E[e- XS ] 



(4.8) 



the value of (4.5) is introduced to evaluate the latter expression. 
Next the variance is computed by subtracting off the square of (4.5). 
In analogous fashion 



E[B 2 (2)] 



1 + 2 E[(l - e XS )]E[B 1 (2)] 
E [e ] 



(4.9) 



insertion of (4.6) and subtraction of its muare yields the variance. 
The covariance is obtained from the expectation 
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_)C _ \ C 

E [ S] + E [ S (1-e ) ]E[ 6 (2) ] + E[l-e A ^ ] E [ t (2) ] 

E[B (2)x (2)] = (4.10) 

1 1 E [e AS ] 

by subtraction of the product of (4.5) and (4.6). These expressions 
can be evaluated in terms of the transform (4.1) and its derivatives, 
and thus there is natural impetus to employ some explicitly transformable 
density, e.g., the gamma or hyperexponential, to represent DTU service 
times . 

Examination of Fig. 3 makes it clear that the busy period moments 
for any J can be expressed in terms of the corresponding moments for 
smaller J-values. This step can perhaps be best carried out numerically, 
for neat closed-form exact expressions will not occur. 

The busy period moments obtained by the procedure described may 
be employed to evaluate the output parameters (3.6) and (3.7). Some 
numerical illustrations are given in the following section. 
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5 . Numerical Examples 

The effect of assuming various parameter values in our multi- 
programming model can be investigated numerically by putting the results 
of the previous section to work. Some rather limited examples appear 
in the following table. 





X: 


0.20 


0.5 


CPU 

1.0 


1.5 


2.0 


5.0 


DTU 




y 


0.17 


0.33 


0.50 


0.60 


0.67 


0.83 


Constant, E[S]=1; J-l; 




a 2 


0.12 


0.15 


0.13 


0.10 


0.07 


0.02 




y 


0.20 


0.45 


0.73 


0.87 


0.94 


0.98 


J=2; 




a 2 


0.18 


0.31 


0.23 


0.12 


0.05 


0.01 




y 


0.17 


0.33 


0.50 


0.60 


0.67 


0.83 


Exponential, E[S]=1; J=1 ; 




a 2 


0.17 


0.19 


0.26 


0.33 


0.35 


0.52 




y 


0.19 


0.43 


0.67 


0.79 


0.86 


0.92 


J=2; 




a 2 


0.17 


0.29 


0.37 


0.47 


0.56 


0.81 


Notice that when A 


Figure 4. 
> (E[S]) _1 = 


1 , in 


which 


case the 


DTU 


stage 



acts as bottleneck, the move from J = 1 to J = 2 has dramatic effects. 
Although the output rate can never exceed unity, improvements of at least 
ten percent occur. The addition of further programs (J > 2) is 
apparently justified only if considerable overhead activity is present; 
this feature is not included in the present model. It is of interest 
to compare the numerical values of Figure 4 to those obtained by Shedler 
[ 4 ] • Clearly A [CPU utilization] = y, and a reference to the 
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appropriate entries in Table 1 of [ 4 ] provides numerical confirmation. 

Examination of the variance of output is of some interest. If 
A is relatively small (CPU the bottleneck) it appears that 

(a) for J = 1, Var[C(t)|s exponential] > Var[c(t)|S constant] 

but 

(b) for J = 2, Var[C(t)|s exponential] < Var[C(t)|S constant] 

By way of explanation, one sees that when J = 2 busy periods are more 
likely to involve more than one DTU service when S is constant than 
when S is exponential. Of course, if A > (E[S]) ^ the DTU becomes 
the bottleneck. As anticipated in this situation the output behaves 
like a renewal process with inter-event times distributed according to 
S. Consequently when S is constant, a 2 = t ^ Var[C(t)] dwindles to 
zero as A increases, reflecting the fact that outputs through the 
DTU bottleneck are regular. Of course, the regularity is even greater 
when J = 2 than when J = 1. If, on the other hand, S is exponential 
the variance gradually approaches that of the DTU bottleneck, namely 
unity. 

It may be guessed from the numbers of the last table that when 
A > 1 the assumption of exponential S provides an underestimate of 
output rate p, and an overestimate of o 2 , provided S is more 
regular — of smaller variance — than the exponential. Another estimate 
of a 2 , useful when the DTU rate is smaller than that of the CPU, 
is obtained by simply assuming that the DTU is never idle, and thus 



a 



2 



Var [S ] 

(eTsJF 
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a familiar renewal theory result that will become increasingly accurate 
for larger and larger J. Although we do not explore such approxima- 
tions further at this point it seems evident that for increasingly 
complex systems — those in which there are considerations of overhead, 
non-exponential distributions, and in which J > 2 — the only practical 
route to understanding is through approximations and bounds. If 
simulations are undertaken it is useful to have some idea of the variance 
of C(t) so that run lengths may be established. Approximate variances 
are often adequate for such purposes. 
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6 . Program Termination and Output 

The previous development takes no account of the fact that 
individual programs actually terminate. In order to introduce this 
effect into the model, we can assume that each time a program leaves 
the CPU stage one of two events occurs: (i) the program terminates 

or is completed, or (ii) the program experiences an honest page fault 
and must go to the DTU stage. Suppose that choice of event (i) or 
(ii) is governed by a Bernoulli trials process so that with probability 
p the program terminates, and with probability q = 1 - p the program 
continues to the DTU stage. In order to allow use of the previous 
analysis we shall assume that in case event (i) occurs a new program 
is immediately introduced into the system at the DTU stage; the first 
pass through this stage may well represent I/O activity on behalf of 
this newest program. 

Let M(t) represent individual program output over time t. Now 
given C(t), M(t) is conditionally binomial, with 

E [M(t) | C (t) ] = pC (t) (6.1) 

SO 

E[M(t) ] = pE [C (t) ] ~ put = p M t (6.2) 

as t °°. Furthermore, 



Var [M(t) ] = pqE [C (t) ] + p 2 Var[c(t)] 

~ pqyt + p 2 a 2 t = a 2 t 



(6.3) 



and since M(t) is easily seen to be a cumulative process the previously 
quoted theorem shows that the actual output of completed programs is 
approximately normal as t 00 . 
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